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ABSTRACT 



Beginning in 1999, an analytical writing measure was offered 
as an optional test in the Graduate Record Examinations (GRE) program. This 
test will be incorporated into the GRE General Test in fall 2002. The essays 
in this test focus on critical reasoning and analytical writing skills. This 
study examined the performance of various examinee groups on this 
performance-based test. Because the test has been optional, the examinees are 
self -selected in that they presumably thought that they would do well on the 
analytical writing tasks. Over the 2 -year period, 5,946 examinee records 
could be matched to both the Writing and GRE General Test. Results provide 
some assurance that incorporating a writing measure into the GRE General Test 
will not necessarily increase group differences. Standardized differences for 
gender groups and many minority groups in this study are smaller on the 
Writing Assessment than on the multiple-choice Analytical measure, which the 
Writing Assessment is to replace. For some groups, most notably examinees who 
may have second language difficulties such as English-as-a- Second- Language 
and Asian American examinees, the standardized differences are larger on 
Writing than on Analytical. From a fairness perspective, these results have 
implications for scoring the assessment. The decision to use the writing 
measure as part of an admissions decision could have a major effect on 
applicants selected for admission. Allowing a choice of Issue topics appears 
to be associated with differential preference rates some of the time. It is 
also apparent that the time allotment for essay tasks has major consequences 
for some examinee groups . Five appendixes contain supplemental material in 
tables. (Contains 13 tables, 12 figures, and 6 references.) (SLD) 
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Performance of Examinee Groups on A Measure of Analytical Writing 



Though admissions tests for graduate school have long used multiple-choice 
questions, they have far less often included performance assessments. Increasingly, 
faculty making graduate admissions decisions have wanted to know more about the 
candidates’ ability to think critically and to express themselves cogently — skills seen as 
essential in most graduate departments. To this end, an analytical writing measure was 
offered as an optional test in the Graduate Record Examinations (GRE) program 
beginning in 1999. (This test will be included in the GRE General Test beginning in Fall 
2002.) The essays in this test focus on critical reasoning and analytical writing skills 
such as the ability to articulate and support complex ideas, to analyze an argument, and to 
sustain a focused and critical discussion. This study will report on the performance of 
various examinee groups on this performance-based test. 



Objectives of the Investigation 

The purposes of the investigation were to examine: 

a) the performance of various examinee subgroups. The primary focus for this topic 
was to determine whether differential performance existed and, if so, to quantify the 
amount of the difference. The groups of interest included male and female 
examinees, minority and White examinees, ESL examinees, and examinees in 
different age groups and in various intended major fields (e.g., Education, 
Engineering, Natural Science). 

b) the amount of differential speededness for examinee subgroups, if any. 
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c) differential selection in the choice of an Issue topic. 

d) the comparative difficulty of different prompt content areas. 

Based on findings from other tests (e.g., Breland, Bridgeman, and Fowles, 1999; 
Breland, Muraki, and Lee, in preparation; Bridgeman and McHale, 1996; Willingham 
and Cole, 1997), the GRE program hypothesized some findings - for example, that 
gender differences would be minimal. However, there were other areas of interest 
for which hypotheses were less clear - such as the performance of groups by intended 
major field. In addition, there was interest in the performance of prompts (e.g., topic 
chosen and time used). 

Description of the Instrument 

The GRE Writing Assessment, which began operational testing on October 1, 1999 as 
a standalone test, was developed to meet the needs of graduate programs for an 
assessment of critical thinking and analytical writing skills. The test contains: 

1) a 45-minute task entitled “Present Your Perspective on an Issue” in which examinees 
state their opinion on an issue of general interest from any perspective and provide 
relevant reasons and examples to explain and support their view. Examinees may 
choose one of two topics presented to them. Choice was offered because the task 
requires examinees to develop the content for their response by drawing upon their 
own reading, observation, and/or experience. 

2) a 30-minute task entitled “Analyze an Argument” in which they critique an argument 
presented to them by discussing how well reasoned it is and considering the logical 
soundness of the argument. 
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These two essay tasks are complementary in that one requires the examinee to construct 
his/her own argument about an issue, whereas the other requires the examinee to critique 
someone else’s argument by assessing its claims and evaluating the evidence it provides. 
All 125 essay topics for each task type are published on the Web in order to facilitate 
examinee preparation. 

When taking the test, examinees may choose to word process or handwrite their 
essays. Each task is holistically scored by two readers on a six-point scale, 1 and the 
scoring rubric is based on the quality of writing demonstrated in the two tasks. Because 
the assessment measures analytical writing, skills such as the ability to reason, to 
assemble evidence to develop a position, and to communicate complex ideas weigh more 
heavily in scoring than does the writer’s control of fine points of grammar or the 
mechanics of writing (e.g., spelling). A single test score is reported on a six-point scale, 
with half-point intervals. The single test score is calculated by averaging the two readers’ 
scores for each task type and then averaging the two task type scores. The final score is 
rounded up to the nearest half point (e.g., 4.25 is rounded to 4.5). 

Information about the reliability of the test and correlations with the General Test is 
contained in Appendix A, and a fuller description of the psychometric qualities of the test 
is contained in Shaeffer, Brief and Fowles (2001), available on the GRE Web site 
(www.gre.org) . 

Description of Sample 

Because the writing measure is relatively new to the GRE program and because there 
is a time lag for departments to decide to require a new test, the examinees in these 
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